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Executive Summary 


This report presents the outcomes of the analysis of Grade Level of Achievement (GLA) data at 
the provincial level to inform the program planning and evaluation needs of Alberta Education as 
well as school and central office based administrators. GLA data reported to Alberta Education 
is a teacher’s judgment of a student’s academic progress. GLA is based on the learner outcomes 
in language arts and mathematics after a course for a specific grade level has been completed and 
reflects the results from the full range of classroom assessments. Given the comprehensiveness 
of classroom-based assessment, analysis of GLA data provides additional insights into factors 
that influence student achievement for students served by a range of specially focused programs. 


GLA data is not used as part of the Accountability Pillar. Key purposes of reporting GLA to 
parents and to Alberta Education include identifying students who are under-achieving, asking 
why and providing solutions both individually and systemically. There are several benefits in 
reporting GLA to Alberta Education including: 

e GLA shows parents how well students are performing when compared to set learning 
objectives. 

e GLA provides schools, school boards and the Province ways to measure the effectiveness 
of education programs targeting special groups like English as a Second Language 
students and students with special needs. 

e GLA results analyzed relative to gender differences, student mobility, and student birth 
month shows how the education needs of students can be better met if these factors are 
shown to be negatively affecting achievement. 

e GLA enables Alberta Education to improve how to assess student performance. 

e GLA requires Alberta Education to be more accountable to Albertans in providing an 
excellent education system. 


Background 


This report briefly describes the processes, and in more depth the outcomes associated with the 
2007-08 Grade Level of Achievement (GLA) data based on the full implementation of the GLA 
reporting initiative. The four purposes for reporting GLA as defined in the GLA Handbook 
(Alberta Education, 2006:4) are: 

e to provide richer information at the system level (both jurisdictional and provincial) to 
inform effective practices to determine the impact of specific programs on student 
learning (e.g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e asa catalyst within the school’s professional learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 


Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


Key Findings from the 2007-08 Analysis 

The degree of school participation in the 2007-08 was 96.2 percent. 

The error rate for data submission was 0.014 percent. 

As expected and similar to the 2006-07 analysis, the 2007-08 data demonstrates less variation 
for the total cohort than for specific sub-groupings of student achievement. For example, 
students achieving At or Above Grade Level in English Language Arts and Mathematics 
differed widely between students coded with severe or mild/moderate disabilities (55 to 63 
percent), Canadian and foreign-born ESL students (77 and 86 percent), non-coded students 
(92 to 94 percent) and lastly, gifted students (98 to 99 percent). 


In addition, the following differences were observed within sub-groups: 


The percentages of students attaining at or above grade level varied substantially by 
specific types of severe or mild/moderate disability. 

Congruent with previously gathered GLA data, nearly three times as many males as 
females were coded as severely disabled and almost twice as many males were coded as 
mildly/moderately disabled compared to females. At the same time, males coded with 
different types of disabilities generally (with a few exceptions) tended to outperform 
coded females on GLA. 

Coded groups of students (especially those with mild/moderate disabilities and Canadian 
and foreign born ESL) tended to perform better in Mathematics than in English Language 
Arts. 

Students coded as gifted were also more likely to achieve above grade level in 
Mathematics. A much higher percentage of these students continue to be assessed above 
grade level in Mathematics than in English Language Arts or French Language Arts. 
Available trend data over two years points to higher percentages of gifted students 
performing above their enrolled grade levels in 2007-08 compared to 2006-07, both in 
English Language Arts and Mathematics. This could be symptomatic either of improved 
programming and more appropriate grade placement of these students. 

When comparing foreign-born and Canadian-born ESL students, both groups of students 
performed at a similar level in Mathematics. With previous (2006-07) GLA reporting 
Canadian-born ESL students appeared to be at an advantage in English Language Arts. 
However, the currently available two-year trend data did not confirm this finding. Recent 
2007-08 data indicate similar percentages of Canadian and foreign-born ESL students 
achieving at grade level in English Language Arts, and indicates foreign-born ESL 
students are closing the gap with their Canadian-born counterparts. 

Slightly higher percentages of mildly or moderately disabled students achieved at/above 
grade level in 2007-08 compared to 2006-07, while their severely disabled counterparts 
showed a reversed trend. However, the comparative trend analysis was based on a partial 
data set (60 percent of schools reported GLA data in 2006-07), which also embraced only 
two consecutive years of data collection. 
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Congruent to findings obtained with previously collected (2006-07) data, females slightly 
outperformed males on GLA (three to five percent more of them were at or above grade level 
compared to males). 


The percent of students below grade level increased through the elementary grades and 
peaked at Grades 8 and 9. Initial two-year trend data revealed subject-based variations in this 
pattern. While the percentage of students below grade level increased with grade for 
Mathematics, the proportion of students decreased with grade level for English Language 
Arts. This finding is compatible with other quantitative and qualitative evidence pointing to 
the issues related to achievement in Mathematics in junior-high. 


Similar to results obtained using past GLA data, the negative effect of high mobility on 
student achievement was evident in 2007-08 GLA results. The differences between highly 
and low mobile students with GLA below grade level ranged from six to over ten percent, 
depending on grade and subject. 


The age effect! was apparent in English Language Arts 2007-08 GLA data, especially in 
Grades 1 and 2. After Grade 3 the age effect tapered off. This relationship also unfolded in a 
similar way in the 2006-07 GLA data, with the only difference that it was more notable for 
Grade 3 and extended to Grade 5. These preliminary trend results confirm that the age effect 
is most apparent in early elementary grades. 


The results of comparisons between GLA and PAT outcomes for Grades 3, 6 and 9 bear 
much similarity to the findings in the previous GLA reporting period. Overall, there was 
almost 80 percent alignment between students assessed at or above grade level of 
achievement and earning acceptable or excelience on provincial achievement tests. The 
previously observed large difference between Mathematics 9 Provincial Achievement Test 
data and GLA (only 65 percent congruence) was also observed in the recent 2007-08 GLA 
data. This phenomenon warrants further trend observations and explanations of why large 
gaps between GLA and PAT assessment results tend to occur in Grade 9 Mathematics. 


Primary purpose of this report 


Using GLA data as a benchmark, jurisdiction and school staff may wish to compare the data in 
school and jurisdiction GLA reports over time or in relationship to provincial norms. This can 
support conversations in professional learning communities and with school councils and parents 
regarding promising practices that have been demonstrated to improve student achievement in 
specific settings or may point to program areas requiring further reflection to improve student 
results. 


' Age effect is defined as older students in a grade tending to have higher average test scores than the younger students in that 
same grade when measured by the z-score of average PAT results fcr each birth month group (Alberta Learning, 2001). 
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Introduction 


This report describes the processes and outcomes for the 2007-08 Grade Level of Achievement 
(GLA) data collection and analysis. The report is also intended to define the provincial context 
supporting the four purposes for reporting GLA as defined in the GLA Handbook (Alberta 
Education, 2006:4) specifically: 

e to provide richer information at the system level (both jurisdictional and provincial) to 
inform effective practices to determine the impact of specific programs on student 
learning (e.g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e asa catalyst within the school’s professional learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 
Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


GLA 2008 Data Collection 


A relatively low number of initial data transmission errors occurred during the 2007-08 GLA 
data collection. Of a total of 359,111 student GLA records submitted, there were 5,145 initial 
errors received (.014 percent). The following table lists the type of error, the frequency and the 
percent of each type of error relative to the total number of errors. 


Table 1 —- GLA Error Counts by Type 


Error Type Number | Percent — | 


Contains both GLA and IPP (Individualized Program Plan) codes for 273 
a single student 


Learner’s ASN (Alberta Student Number) not found | 45 
Incorrect GLA or IPP code. | 12 =| 023 | 


Grade English Language Arts Introduced column incorrectly filled in 
for French Language Arts students. 


School code does not belong to authority code or school year is | 1 | oe 
wrong. 


Duplicate record of learner (i.e. two entries for same person). | 600 | 11.66 


Student not registered at specified school; enrolled grade does not 3,868 75.18 


match grade registered in; or ASN is retired. 


Total | 5,145 | 100.0 


For all errors a Datacheck error file, by type, was created and returned to the source jurisdiction 
for correction and resubmission. The resubmitted file was reloaded and if errors were again 
found the Datacheck error file process was repeated until it was deemed unnecessary to continue 
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returning error files and the data for that jurisdiction was deemed error free or as error free as 
possible. 


The corrected individual student records have thus been included in the 2007-08 data throughout 
this report as well as in the school and jurisdiction reports available on the Extranet. One school 
authority was advised of two student cases where the specific errors were irreconcilable so they 
could accommodate this discrepancy in their interpretation of the value added GLA reports 
provided back to the jurisdiction and school that submitted GLA data. Alberta Education will 
consider front-end data cleaning procedures to eliminate the need to clean data post-submission. 


Design and Description of GLA Data 


A total of 1602 schools from 72 authorities submitted usable Grade Level of Achievement data, 
reporting for 353,815 students, 4,927 of whom were not on a graded curriculum. The fields 
collected are as follows: 


All Students: 

e student name (surname and given name), 

e Alberta Student Number, and 

e enrolled grade (defined as the grade to which the student was assigned). 


GLA was collected for students on a graded curriculum as defined in the Alberta programs of 
study, in the following fields where applicable: 


e GLA in English Language Arts 

e GLA in French Language Arts - (French as the Language of instruction or Immersion 
students) 

e GLA in Mathematics 

e Grade English Language Arts Introduced (for French Language Arts students only) 


Grade Level of Achievement in 2007-08 is reported as the following three categories: (1) GLA at 
grade level; (2) GLA above grade level; and (3) GLA below grade level. A GLA Handbook 
(Alberta Education, 2006) was updated and revised in January 2008 to facilitate data submissions 
for the 2007-08 school year. The GLA Handbook encourages teachers to consider GLA 
assessment in relationship to the full range of formative and summative assessment information 
available to them over the course of the school year in making a professional judgment of the 
student’s grade level of achievement. 


Students not on a graded curriculum also had data submitted. “Not on a Graded Curriculum” 
was meant to indicate that the student’s program was restricted to learning outcomes that were 
significantly different from the provincial curriculum defined in the program of studies and were 
specifically selected to meet the student’s special needs as defined in the Standards for Special 
Education (Alberta Learning, 2004). The information collected was teachers’ ratings of 
students’ learning outcomes in three areas: communication skills, functional skills and academic 
readiness skills. “Communication skills” refer to the development of expressive and/or receptive 
communication. This could be verbal communication and/or alternative modes of 
communication. “Functional skills” refer to skills that would assist the student in developing 
independence in the home, school and community. “Academic readiness skills” refer to skills 
that would prepare the student for learning outcomes in the programs of study. 


Alberta Education staff used the Alberta Studert Number to append data fields such as 
Provincial Achievement Test (PAT) results (both raw scores and achievement levels), student 
age, gender, number of school registrations, any additional special needs codes associated with 
the student, and the student’s school starting date. Individual student identifiers were replaced 
with a discrete GLA data ID, leaving no personal identifiers in the dataset used in producing this 
report. 


Grade Level of Achievement — Summary of Results 


Students on a Graded Curriculum 


There were 348,888 students on a graded curriculum included in this report for the 2007-08 
school year. The students are roughly evenly distributed by enrolled grade with approximately 
11 percent of the students in each grade cohort. The table below shows the distribution of the 
GLA sample data by enrolled grade (Grades | to 9). 


Table 2 - Enrolled Grade Distribution 


Enrolled GLA Frequency 


Grade Percent of GLA Total Province Percent of Total 
37,525 10.8 39,724 10.7 

2 36,690 10.5 38,834 10.5 
3 37,749 10.8 39,896 10.8 
4 38,347 11.0 40,388 10.9 
5 38,246 11.0 40,329 10.9 
6 39,370 113 41,576 11.2 
7 40,515 11.6 42,967 11.6 
8 40,273 T1.5 43,071 11.6 
9 40,173 ILS 43,388 17 

Total 348,888 100.0 370,173 100.0 


When compared to the number of students registered in Grades 1-9 there is a discrepancy of 
21,285 students or 5.8 percent. This shortfall was due to the non-reporting of GLA by 63 
schools providing programs to Hutterite students, distance education programs, students in 
blended programs, other specialized programs, and a few schools that missed the deadline for 
submitting GLA data. The information regarding discrepancies relative to GLA data received 
vs. student registrations was referred to Field Services in Alberta Education for follow-up with 
the appropriate central office and school staff. The objective is to ensure that GLA is reported 
for 100 percent of Grades 1-9 students enrolled in public, separate, Francophone and charter 
schools by the school that holds the primary registration for the student. 


Students may be coded as severely disabled, mild/moderately disabled, gifted or ESL. Students 
who were not coded as any of the preceding categories are termed ‘non-coded’. The non-coded 
students make up the largest proportion of the sample as shown in Table 3 below. 


Table 3 — Types of Student Codes 


GLA Percent of Province Percent of 
Frequency GLA Total Provincial Total 

peg mild/moderate, severe, 276,521 793 287,997 77.8 
Severe Disabilities 9,188 2.6 12,880 3.5 
Mild/Moderate Disabilities 24,791 7.1 28,427 7.7 

Gifted 5,050 1.4 5,193 1.4 

ESL —Canadian-born 18,595 5.3 20,510 5.5 

ESL — Foreign-born 16,859 4.8 18,700 5.1 

Total 351,004* 373,707* 


*This is higher than the total in Table 1 because some students have multiple codes. 


The total distribution of students in each of the GLA results categories by subject is shown in 
Table 4 below and the results for the cohort of non-coded students are displayed in Table 5. 


Table 4— GLA Results for All Students, Provincial 


a English Language Arts French Language Arts 


students total enrolled students total enrolled students total enrolled 
otawa? «saat a —*| se «| orto 


Table 5— GLA Results for Non Coded Students at a Provincial Level 


Mathematics English Language Arts French Language Arts 
Number of Percent of Number of Percent of Percent of 
students total enrolled students total enrolled total enrolled 


Number of 
students 
’ 


5.8 5.8 


Students Coded with Severe Disabilities 
There were 9,188 students coded as severely disabled who had GLA reported for English 
Language Arts (ELA) and Mathematics. There were 155 students coded as severely disabled 


> GLA N/A refers to missing data, “not applicable” situation, or “not available.” 


who had GLA reported for French Language Arts (FLA). Table 6 presents students having a 
severe disability code and their grade level of achievement. 


Table 6 — GLA Results for Students with Severe Disabilities at a Provincial Level 


Mathematics English Language Arts French Language Arts 
Number of Percent of Number of Percent of Number of | Percent of total 
students total enrolled students total enrolled students enrolled 


GLA N/A ee ; 


Over half of students with a severe disability enrolled in Mathematics have a GLA equal to their 
enrolled grade (56.7 percent). In English Language Arts 54.5 percent of students have a GLA 
equal to their enrolled grade. Mathematics and English Language Arts are fairly similar in their 
GLA distribution. A very low proportion of French Immersion or French as the language of 
instruction students were coded as severely disabled. 


Table 7 provides GLA information on students on a graded curriculum who were coded as 
severely disabled, by their disability type. The majority of students coded with a severe 
disability are those with an emotional/behavioural or physical/medical disability. These 
particular groups of students, along with small groups of students with severe multiple 
disabilities and deafness, exhibited low GLA outcomes. The proportions of these students 
assessed below grade level ranged between 34 and 54 percent in Mathematics and were even 


Table 7 —- GLA Results by Type of Severe Disability 


Type of Severe At/above | Below grade | GLA | At/above Below GLA 
Se ctciis Total rade level level NA rade level | grade level 
Disability Frequency g' g g 


Percentage 
Percentage 


Severe Cognitive 


Severe Emotional 
/Behavioural 


Severe Multiple 


higher in English Language Arts. At the same time, as high as 77.4 percent of students with 
blindness met the grade requirements in Mathematics and only 19 percent were below grade 
level. The high degree of variation in GLA results across the different types of severe 


disabilities demonstrates the importance of considering the specific types of disability in 
unpacking achievement results for students with severe disabilities as well as for planning to best 
meet students’ learning needs. 


When the students coded as having a severe disability are split by gender, the large 
difference between genders becomes apparent (see Table 8 below). While an overwhelming 
majority (77.9 percent) of students coded with a severe disability were males (most of 
whom were students with an emotional or behavioural disability), severely disabled males 
tended to generally perform better then severely disabled females. Higher percentages of 
males across most disability types had GLA at or above grade level in both Mathematics 
and English Language Arts compared to females with the exception of students who are 
blind. 


Table 8 — GLA Results for Students with Severe Disability by Gender 


g P 
Disability | At/above y GLA At/above y GLA 

code grade NA grade NA 
(percent) level level 


Percentage 


Type of 
Severe 
Disability 


Severe 
Cognitive Female 


/Behavioural 


Physical or 


Multiple 

Paste fee wf ae os for a | se fa 

ee 
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Students Coded with Mild/Moderate Disabilities 


There were 24,791 students in 2007-08 having mild or moderate disability codes in English 
Language Arts and Mathematics. In French Language Arts, a small group of 699 students were 
coded as having a mild or moderate disability. Table 9 below shows these students’ distribution 
across GLA categories. 


In both Mathematics and English Language Arts, the proportion of students with mild or 
moderate disabilities who have a GLA equal to their enrolled grade hovered around 60 percent 
and the proportion of students with GLA below enrolled grade varied between 35 and 40 percent. 
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These general results for students coded as having mild or moderate disability are not dissimilar 
to the corresponding GLA outcomes for students identified as severely disabled (see Table 6). 


Table 9 - GLA Results for Students with Mild/Moderate Disabilities 


English Language Arts French Language Arts 


Percent of Percent of Percent of 
Number of Number of Number of 
total total total 
students students students 
enrolled enrolled enrolled 


Fataatoveciteagnde | 90 [oa | @ | 03 | «| 0s | 
Prot Sirona 


When split by the type of disability (refer to Table 10), GLA results for students on a graded 
curriculum coded with a mild or moderate disability reveal a lot of variation depending on 
disability category. The largest proportion of students had a learning disability, followed by 
communication disability and mild cognitive disability. The highest proportion of students 
attaining below their grade level (64 to 68 percent) belonged to the groups coded with a mild or 
moderate cognitive disability. Only a quarter or less of these students reached or exceeded grade 
level of achievement. At the same time, a small number of students with a mild or moderate 
hearing or visual disability tended to achieve at a relatively high level, with about 80 or higher 
percentages performing at or above grade level. By analogy, relatively high proportions (close to 
two-thirds) of students with mild/moderate emotional/behavioural, communication or 
physical/medical disabilities achieved at or above GLA in both Mathematics and English 
Language Arts. 


Table 10 - GLA Results by Type of Mild/Moderate Disability 


Type of At/Above At/Above 
Mild/Moderate Percent Grade Grade GLA NA 
Disability* Frequency | or Total Level Level 

Percentage 


Paiacopve | see [a7 _| m7 | 9 | 44 | 58 | os | 06 | 
[Moderate Comiive [130 [06 [ais] oo | 147 | 200 | 3 | 167 
E 


: 

oome | ta | ae | oo | a7 | aa | me | ae | 6 
Vinal [| # [02 [ma [a2 | oo | mi | iso] 00 
PhyscalMedil [14 [60 [4s | ma | 1a | ma | 259 | 19 
From | on ie | ee | ass [19 [| sa | wa | 24 


*Two students enrolled in Grade 1 coded as ECS Developmentally Immature and ECS Mildly or Moderately Disabled were removed from this 
analysis. They have been included in the rest of the report. 


Table 11 below illustrates the gender differences in GLA categories across the mild/moderate 
disability types. In all, by analogy with students with severe disability codes, coding in various 
mild/moderate disability groups was noticeably prevalent among males — consistently higher 
percentages of males were coded in this category compared to females (see the “ percent of each 
disability” column). There was also apparent gender-based difference in GLA results depending 
on the type of mild/moderate disability. For example, these gender differences are most 
noticeable for students having an emotional/ behavioural and physical/medical disability. 


Table 11 - GLA Results for Students with Mild/Moderate Disability by Gender 


Mathematics 
Freq 


uency 
124 


Type of Mild Each At/Above GLA At/Above 
or Moderate Gender Disability Grade NA Grade 
Disability* (percent) Level Level 


Percentage 
Medical : 


Male 
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ns 
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Mild Cognitive 


ea 
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*Two students enrolled in Grade 1 were coded as ECS Developmentally Immature ECS Mildly or Moderately Disabled and were removed from 
this analysis. They have been included in the other sections of the report.) 


Mae [29 [659 | 793 | 207 | oo | 793 | 207 | 


In a similar way to the results for students with severely disabled coding (Table 8), generally 
lower percentages of females in the mildly/moderately disabled group reached or exceeded the 
grade level of achievement compared to males (Table 11), although these differences tended to 
be less pronounced than among the students with severely disabled codes. 


Gifted Students 

Among 348,888 students on a graded curriculum with GLA data available, 5,050 were coded as 
being gifted in English Language Arts or Mathematics. There were 340 students in French 
Language Arts coded as gifted. Table 12 shows the grade level of achievement distributions for 
these students. 


Table 12 - Gifted Students, Provincial 


Mathematics English Language Arts French Language Arts 
Number of Percent of Number of Percent of Number of Percent of total 
students total enrolled students total enrolled students enrolled 
GLA below enrolled 
grade 
OLA cnet ie carelie 84.1 4,676 92.6 266 78.2 
grade 
grade 


a eo 


The general assumption with gifted students is that they tend to achieve and perform better than 
the population of students as a whole. According to the above frequency table, we can see that 
the vast majority of gifted students were performing equal to their enrolled grade level (84.1 and 
92.6 percent in Mathematics and English Language Arts respectively), and under one percent of 
students were “below grade level” for each subject. Gifted students were performing better in 
Mathematics than in English Language Arts with approximately nine percent more students 
having a GLA “above grade level” in Mathematics. 


English as a Second Language Students 


For English as a Second Language (ESL) students, there were two groups of student codes. The 
first are ESL students coded as Canadian-born (code 303). Of this group of students there were 
18,595 in English Language Arts and Mathematics and 905 students in French Language Arts. 
For ESL students who are coded as foreign-born (301), there were 16,859 in English Language 
Arts and Mathematics and 551 in French Language Arts. The distribution of both groups’ GLAs 
is shown in Tables 13 and 14 below. Both Canadian-born and Foreign-born ESL students follow 
a similar GLA pattern. Approximately 83 to 84 percent of each ESL group were attaining a GLA 
that was equal to or above enrolled grade in Mathematics and about 76 to 77 percent was equal 
to or above grade level in ELA. 
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Table 13 - Canadian-born ESL Students, Provincial 


SS _.- English Language Arts French Language Arts 


Number of Percent of Number of Percent of Number of Percent of total 
students total enrolled students total enrolled students enrolled 


grade 
[awa [vases [awa [ws | sono 


Mathematics English Language Arts French Language Arts 
Number of ay of Number of Percent of Number of Percent of total 
students students total enrolled students enrolled 
enrolled 
grade 


GLA equal to enrolled 14.224 84.4 12.763 
rade 
rade 


Student Gender 

Given growing interest in achievement differences depending on gender, students’ GLA was 
analyzed by gender in order to observe any anomalous patterns that may emerge. In 2007-08 
data there was a somewhat larger number of males than females. Tables 15 through 17 show 
students’ GLA by gender, with females consistently, but slightly, outperforming males in 
Mathematics, English Language Arts, and French Language Arts. 


Table 15 - Gender, Mathematics, Provincial 


Eee 
students enrolled students total enrolled 
|GLA below enrolled grade | 13,576 =| 8.0 | t8,007 | tt 


GLA above enrolled grade 2,635 2,897 
GLA NA Ze 
Total 170144 [| __ 100.0] 178,744 | 100.0 


1] 


Table 16 - Gender, English Language Arts, Provincial 


Pe oe 
students enrolled students total enrolled 
[GLA below enrolled grade | 12.909 | 76 227s 


7.6 

GLA equal to enrolled grade 149,652 149,703 
GLA above enrolled grade 1.7 
: 7 


GLANA 4.676 5050 
i7oaaa [1000 i774 | 1000 


Table 17 - Gender, French Language Arts, Provincial 


| Female | Male 
Number of Percent of total Number of Percent of 
students enrolled students total enrolled 


GLA below enrolled grade 
1:9 


GLA equal to enrolled grade 12,429 9,766 
GLA above enrolled grade 


GLA NA 1503 1,204 
14,720 | 000 | 14,74 100.0 


Student Grade 


As illustrated in Table 18 below, the percentages of students who did not meet their grade level 
of achievement tended to increase with grade in Mathematics. The results for English Language 
Arts revealed a reversed trend: somewhat higher percentages of older students were more likely 
to achieve at grade level in comparison to younger students. Proportions of students above grade 
level are very similar across the grades, and proportions of students at grade level are lower for 
older students with the biggest achievement decline between Grades 8 and 9. 
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Table 18 — GLA by Grade, Provincial 


Mathematics 


GLA below 
enrolled grade 


GLA equal to 
enrolled grade 


GLA above 
enrolled grade 


268 


Total 100.0 | 37,525 A . . A 100.0 | 38,347 38,246 | 100.0 | 39,370 


English Language Arts 


GLA below 
enrolled grade 


ee oar 78.0 oy 2 | 35,745 | 88.3 
see [el | = oc 
enrolled grade , ; , ; : 
Pela | 
om [| 


11.0 4,124 ; : ; : ‘ 10.5 
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Student Mobility 


Changing schools, especially during the school year can be disruptive for a student who may be 
placed at a different point in the curriculum than he/she left behind. It also takes some time for 
teachers to determine the student’s learning level, learning style, interaction skills, etc., and thus 
define the optimal program. 


These issues concur with the findings within the research literature of a negative relationship 
between the number of times a student changes schools in a given period and his/her academic 
growth in that period. Other Alberta studies (Wasserman, 2001) of this relationship have 
supported these findings and suggest that additional research would be useful. Further 
investigation would enrich our understanding of the relationship between student mobility and 
academic achievement while highlighting any situations in which the negative impacts may have 
been mitigated by helpful strategies to support better transitions for students. 


Student school registrations are captured by Alberta Education, once at the end of September and 
again in March, and compiled in the Student Information System (SIS). The Student Mobility 
Indicator (SMI) provides an indication of the number of times a student has changed schools 
since entry into the Alberta school system. The SMI is calculated by counting the number of 
different school registrations each student has up until the most recent calendar year. Students 
could be changing schools more frequently than is captured, thus the numbers shown may be a 
conservative estimate of student mobility. All students start with an SMI of 1 as they have all 
been registered in at least one school. Student mobility is then broken down into two categories 
— high and low. In Grades 1-3, high mobility students are those having a mobility indicator of 2 
or more. Students having a mobility indicator of 1 are considered low mobility. In Grades 4-6, 
high mobility students are those having a mobility indicator of 3 or more. Students having a 
mobility indicator of 2 or less are considered low mobility. In Grades 7-9, high mobility students 
are those having a mobility indicator of 4 or more. Low mobility students have a mobility 
indicator of 3 or less. In the following tables (Tables 19 through 21) the two categories of 
mobility include students on a graded curriculum in Grades | to 9. 


Table 19 - Student Mobility, Mathematics, Provincial 


High Mobility Low Mobility 
Mathematics 


students enrolled students enrolled 
[LANA | uaa «df «Sinise «YS 
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Table 20 - Student Mobility, English Language Arts, Provincial 
students enrolled students enrolled 
row SSCS~C~wCtwa tow ase | to 


English Language Arts 


French Language Arts students enrolled students enrolled 
FGtAtelowewolledaade | 196 | 38 «| a7 
row —SSCS~sSSC «St 


The majority of students can be described as having low mobility (72.2 percent of all students 
with GLA data in 2007-08 compared to 69.1 percent in 2006-07).> As expected, a greater 
proportion of high mobility students have a GLA below their enrolled grade level compared to 
low mobility students. 


Tables 22 and 23 further examine the effect of student mobility on grade level of achievement in 
English Language Arts and Mathematics. The graphs depicted in Figures | and 2 and based on 
the data shown in Tables 22 and 23 illustrate that there is a notable difference between high and 
low mobility students’ GLA outcomes in both English Language Arts and Mathematics. 
Consistently, higher percentages of highly mobile students achieved at below grade level 
compared to their counterparts who displayed low mobility. 


* There is no mobility information on 14,509 students (4.2%) in the 2007-08 GLA data set. 
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Table 22 - Student Mobility and English Language Arts by Enrolled Grade 


Ee ee 668 i 7 1 ES | an 9 SCae eee 
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Table 23 - Student Mobility and Mathematics by Enrolled Grade 
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Figure 1 


Percent of Students Below Grade Level by Mobility Category for English 
Language Arts (2007- 08) 


» —e~High Mobility 
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Percent Below Grade Level 
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The graph illustrates the effect of student mobility on achievement in English Language Arts by 
contrasting the percentages at below grade level for students in the high mobility and low mobility 
categories. Consistently, throughout all nine grades, about twice as many highly mobile students (13 
to18 percent) achieve below grade level in English Language Arts compared to low mobility students (6 
to10 percent). 


Figure 2 
Percent of Students Below Grade Level by Mobility Category for 
Mathematics (2007- 08) _— High Mobility 
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This graph juxtaposes the percentages of students distinguished by high mobility versus those with low 
mobility below grade level in Mathematics. Congruent to English Language Arts, approximately twice 
as many students in the high mobility category are more likely to achieve below grade level in 
Mathematics (11 to 20 percent) in comparison to low mobility students (six to 10 percent). 
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Students not on a Graded Curriculum (Modified Programming) 


There were 4927 students reported in 2007-08 who were not on a graded curriculum. These 
students were assessed based on the degree of achievement in their Individual Program Plans 
(IPPs) relative to their foundational skills, academic readiness skills and life skills. The results 
reported below should be considered preliminary as data will become more meaningful as multi- 
year trends become available. 


Table 24 - IPP Foundation Skills 


Number of Students | Percent of Total Enrolled 


Table 25 - IPP Academic Readiness Skills 


Table 26 - IPP Life Skills 


Life Skills Number of Students | Percent of Total Enrolled 
All skills attained 


Most skills attained 


Some skills attained 


None of the skills attained 


The available data demonstrate that consistently, across all three skill categories, about 40 
percent of IPP students tended to attain all or most of these skills and about one-third mastered 
some of the skills. 
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Birth Month-Combined Grades 


Table 26 shows the distribution of students on a graded curriculum by birth month for all grades 
of students. Students in the 1“ January and February are the oldest students in each grade, 
whereas students in the 2" January and February are the youngest in each grade. 


Table 27 — Student Birth Month 


Fitamay [18956 | 61 
[i Fetray [1950 | 63 
PMc «ato | oa 
Fo 
a a 
Pimento 
a 
[a 
ra 


6.1 
6.3 
9.4 
9.1 
Proie Sisson | i000 


*Not all students were included in this table since some students fell outside the scope of analysis meaning they were much older 
than the targeted group of students. 


Previous research (Alberta Learning, 2001) based on provincial achievement test (PAT) scores 
demonstrated that older students tended to achieve at a higher level than their younger 
counterparts. A comparative analysis, which was undertaken using 2006-07 GLA data 
confirmed a moderate positive relationship between student age and GLA outcomes in English 
Language Arts in Grades | through 5. After Grade 5, the age effect tapered off and was no 
longer apparent. A similar analysis was repeated using more recent and complete GLA data 


from the 2007-08 reporting cycle and results of these analyses are discussed in the following 
section. 


GLA and PAT Results by Age within Student Grade Cohorts 


The percentages of students “at or above” their grade level in English Language Arts were 
converted and plotted on the graphs shown below. In general the results confirmed previous 
findings that younger students in early grades are more likely to attain lower academic 
performance compared to their older peers. The data on Grade | and 2 students (Figures 3 and 4) 
showed evidence of age effect, which was much less pronounced in Grade 3 GLA data (Figure 5). 
However, Grade 3 PAT data revealed notable differentiations in student achievement depending 
on the birth month. The graph in Figure 6 demonstrates that when PAT scores are recoded into 
“percent at or above acceptable” to mimic the GLA data, the relative age effect remains. 
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Congruent with the findings based on the previously collected (2005-06) GLA data, 2007-08 data 
did not reveal a notable age effect in Grades 4, 5 and older. The implication of these findings is 
that schools with pronounced age effects may want to take this into account in interpreting Grade 
3 PAT results and may consider strategies for diminishing the age effect. Also, the case can be 
made for multiple entry points into the ECS and Grade | programs and for careful consideration 
of the extent to which the age effect might influence student achievement within a specific 
classroom or school context. 


Figure 3 


Grade 1 English Language Arts: At/Above GLA | 
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This graph shows percentages of students who were at or above grade level in Grade 1 relative to 
birth month. The straight line represents a linear trend. Students with birthdays falling on the 1* 
January/February were the oldest in the cohort and students with birthdays on the 2™ 
January/February were the youngest. There was a notable age-based difference in student 
achievement in Grade 1 with the oldest students outperforming the youngest ones by approximately 
five percent. 
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Figure 4 


This graph shows percentages of students meeting their grade level in Grade 2 depending on the 
month of birth. In general, similar to Grade 1 data, higher percentages of older students with earlier 
birth month were achieving at or above grade level compared to their younger counterparts. 


Figure 5 
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Grade 3 sss Language Arts: At/Above GLA | 
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By Grade 3 the age effect was much less apparent in 2007-08 GLA English Language Arts data. 
There is not much difference between the older students (the oldest ones with birthdays falling on the 
1“ January/February) and their younger peers (with birthdays in the 2" January/February). In all, 
this finding is congruent with preceding research, which indicates that the age effect tends to be 
more pronounced in the earliest elementary grades. 
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Figure 6 
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Unlike Grade 3 GLA data, Grade 3 PAT data plotted on this graph (percent of students at or above 
acceptable level of achievement) show more pronounced differences in student achievement 
depending on the month of birth. Approximately seven percent more of the oldest students who 
were born in the 1° January/February achieved at or above acceptable level compared to their 
younger counterparts born in the 2"’ January/February. The differences in GLA and PAT data 
may be attributed to their different nature and data collection methodologies. Variations like this 
illustrate the usefulness of holistic approaches to data collection and analysis, in order to capture 
diversified trends and issues in student academic achievement. 
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Dynamics in GLA Data Between 2006-07 and 2007-08 Reporting 
Periods 


The following section of the report is based on the two last years of GLA data collection and 
highlights the type of trend analysis that will become viable as the GLA initiative becomes 
implemented over several years. The data in these two data sets were matched to capture the 
same students and follow their progress in two GLA reporting years. The analysis was run on 
the same matched 179,355 students that were usec for trend analysis of non-coded and coded 
students. 


Table 28 below shows general GLA results for all matched students. There is an indication of a 
slight increase in the percentages of students below grade level for Mathematics in 2007-08 
compared to 2006-07, but a decrease of students in this category for English Language Arts. 
However, these results should be considered as preliminary and illustrative, since the 2006-07 
data set was incomplete due to the pilot status of GLA reporting that year and a relatively high 
proportion of cases had the data classified as “GLA N/A,” indicating missing (“not available”) 
data. 


Table 28 — Difference in GLA Results: All Matched Students in 2006-07 and 2007-08 
GLA Data Sets 


2006-07 2007-08 
Number of | Percent of total | Number of Percent of 
students matched students total matched 
Mathematics 


GLA below enrolled grade 15,743 lz ia 16,309 


‘i 8.8 
Towa Machedin 300607200708 | 19ass_| 19.0 | 


GLA below enrolled grade 
Total Matched in 2006-07 — 2007-08 


Tables 29 and 30 display the same data split by grade. These data are also plotted in Figures 7 
and 8. It is important to keep in mind that a relatively small proportion of the reported students 
might not have advanced an enrolled grade from 2006-07 to 2007-08. For example, 426 of 
2006-07 Grade 1 students were also in Grade 1 in 2007-08 (see Tables 29 and 30), and 58 
students who were in Grade 9 in 2006-07 also were present in the 2007-08 Grade 9 group. 


179,355 
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Overall, the data reveal somewhat higher percentages of students below grade level up to Grade 
6 for English Language Arts compared to Mathematics. After Grade 6 this trend reverses, with 
slightly higher proportions of junior-high students achieving below grade level in Mathematics 
compared to English Language Arts (juxtapose percentages in low columns in Figures 7 and 8). 
Put another way, while the percentage of students below grade level tended to increase with 
grade for Mathematics (from six to seven percent in Grade | to 12 percent in Grade 9), the 
proportion of students in this category tended generally to decrease with grade for English 
Language Arts, with a peak at around upper-elementary level (11 tol3 percent in Grades 4 
through 6 and a dip at a junior —high school (eight to nine percent). 


Table 29 — Comparisons of GLA in Mathematics in 2006-07 and 2007-08 by Grade 


2007-08* 


2006-07 1.273 


Above enrolled 
grade 


Below enrolled 
a 


Equal to enrolled 
grade 


Total matched 
cases by grade 


GLA N/A 


Percent Percent 
92.0 


88.0 8 1.9 


Percent 


20,279 


Percent 
eee 


4 0.9 


2006-07 


1,484 20,559 92.0 0.3 


BY 5s 
cao 
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2006-07 
2007-08 


2006-07* 
2007-08 


Total 


2006-07 | 15,743 160,349 | 89.4 | 792 | 04 | 
2007-08 | 16,309] 9.1 | 159,201 | 88.8 | 2388 | 13 


Tas aa | 
2006-07 | 2,000 20,092 | 898 | 74 eee 
2007-08 | 1,821 20,189 | 903 | 245 0.5 | 22,362 

2006-07 2330 |) 104 | 49769 | a2 | 2 || Hote ets 


2007-08 | 2,083 81 | 885. |- 256 
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36.2 
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48.3 
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2,332 20,271 88.7 53 0.2 0.8 22,846 
2,156 | 9.6 | 19,889 | 88.7 251 1 0.6 | 22,423 
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*N’s for Grade 1 in 2007-08 and Grade 9 in 2006-07 are low as these students are repeating the grade and are 
therefore not included in Figures 7 and 8. 
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Table 30 — Comparisons of GLA in English Language Arts in 2006-07 and 2007-08 by 


Grade 
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Gr. 4 19,647 


rao07-08 [2376 | 106 | 


Total 


Gr. 7 


Below enrolled | Equal to enrolled 
grade grade 
TN [Percent | N| 


N 
oe a ee eee 


i (oma |e [se a 
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grade 


a 
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0.6 | 22,362 | 100 
1.4 | 22,407 | 100 
| 09 | 135 | 06 | 22,423 | 100 


raon6-07 [isos | 106 | 1saasi [ao | ae | 02 | same | 34 [179355] 100 | 


Total matched 
cases by grade 


GLA N/A 


6.5 | 22,353 | 100 
21,721 | 100 


*N’s for Grade 1 in 2007-08 and Grade 9 in 2006-07 are low as these students are repeating the grade and are 


therefore not included in Figures 7 and 8. 
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Figure 7 — Comparisons of GLA in Mathematics in 2006-07 and 2007-08 by Grade 
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The graph depicts the dynamics of GLA Mathematics data collected over two years (2006-07 and 2007-08). Each column represents 100 
percent of all students that were matched across the two GLA reporting years. The lower section of the column shows percentages of students 
who were below their enrolled grade. The middle portion of the column shows proportions of students who were at grade level, and the upper 
sections show percentages of students above grade level and/or percentages of students with no GLA data (GLA N/A). While the vast majority 
of students across all nine grades (over 85 percent) were at/above grade level, the percentages below grade level tended to increase with grade, 
almost doubling from six to seven percent in Grade 1 to 12 percent in Grade 9. This trend clearly reflects province-wide issues with student 
achievement in mathematics. GLA data provides a supplementary, more consistent base for monitoring and addressing this issue compared to 
sporadic PAT tests. 
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Figure 8 — Comparisons of GLA in English Language Arts in 2006-07 and 2007-08 by Grade 
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The graph depicts the dynamics of GLA English Language Arts data collected over two years (2006-07 and 2007-08). Total column represents 
100 percent of all students that were matched across the two GLA reporting years. The lower section of the column shows percentages of 
students who were below their enrolled grade. The middle portion of the column depicts proportions of students who were at grade level, and 
the upper sections show percentages of students above grade level and/or percentages of students with no GLA data (GLA N/A). Unlike GLA 
outcomes for Mathematic, the proportion of students below grade level in English Language Arts tends to reach the highest point in upper- 


elementary grades and then decrease in junior-high school. 
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The following section of the report compares the two last years of GLA data on coded and non- 
coded students on a graded curriculum. These comparisons were run in order to demonstrate 
potential practical utilities of the future GLA data collections for identifying long-term trends 
and programming issues and hence, clearer directions for remedial interventions. 


Comparison of Outcomes for Non-coded and Coded Students in 2006-07 and 2007-08 


A data set comprising 217,302 students on a graded curriculum were collected during the 
previous 2006-07 GLA reporting cycle. However at that stage of GLA reporting, not all Alberta 
public, charter, separate and Francophone schools were required to submit GLA results. 
Therefore, although sizeable, the resultant data set is not complete and does not fully reflect the 
Alberta student population. The most recent GLA data collection yielded usable data on 348,888 
students and came much closer to fully embracing student population in the mentioned types of 
schools. 


As noted earlier, the data in these two data sets were matched to capture the same students and 
follow their progress in two GLA reporting years. As a result, 179,355 matching student cases 
were identified and constituted the base for the data analyses reported below. A relatively 
“small” number of matched cases may be explained by the vast majority of 2006-07 Grade 9 
students moving to Grade 10 and Early Childhood Services (kindergarten) students entering 
Grade 1 in 2007-08 and hence not being included in the 2006-08 GLA reporting frame, which 
embraces Grades | through 9 students only. In addition, some 2006-07 students have migrated 
out of the province, joined a private school or for other reasons are not registered in a public, 
charter, separate or Francophone school. 


In order to compare the progress of similar groups of students, students who were identically 
classified as non-coded, mildly/moderately or severely disabled, gifted or ESL in both 2006-07 
and 2007-08 GLA reporting periods were selected for the analysis highlighted in Tables 31 and 
32 below. The analysis was performed on generalized data on Mathematics and English 
Language Arts, without breaking the results by grade. Since this analysis was run on “truncated” 
matched data sets, some results may differ from an analogous analysis, which was run using the 
full 2007-08 data set. 


The data on the matched non-coded students in Table 31 do not reveal any major differences in 
the proportions of students at or below enrolled grade between the two GLA reporting periods, 
and, in fact, demonstrate a high degree of consistency. The differences hover around half to one 
percentage point. For example, slightly higher proportions of students achieved at grade level in 
Mathematics in 2006-07 compared to the 2007-08 school year, but, reversely, somewhat higher 
percentages finished at grade level in English Language Arts in 2007-08. In addition, about one 
percent more of the examined students were above grade level in 2007-08 than in 2006-07. 
However, consideration of these results should take into account the higher percentages of 
missing data (“GLA N/A”) in 2006-07, which might incorporate some of the students for whom 
the GLA data became available later, in 2007-08. 
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Table 31 — Difference in GLA Results: Matched Non-coded Students for 2006-07 and 
2007-08 


2006-07 2007-08 


Non-coded Students (as mild/moderate, severe, gifted, or ESL) 


FGLAtwlowenrcagade | 46 | oio7| si [sous | so | 67m | a7 | 63s 
2 


All Grades 


4.7 
GLANA Sido | gos P23 | sume [os | om | [ar | 


Total for Matched Students in 
2006-07 — 2007-08 100 | 134,840 100 | 134,840 100 | 134,840 134,840 


Table 32 includes findings on coded students. Surprisingly, the GLA for students coded as 
having severe disabilities was not much different from students who were classified as mildly or 
moderately disabled, with over 50, but under 60 percent of the students in both categories 
achieving at the grade level in both reporting periods, about 40 percent falling behind their grade 
level and 0.5 percent or under being assessed above their enrolled grade. Nevertheless, while 
both Mathematics and English Language Arts GLA results were slightly better in 2007-08 than 
in 2006-07 for mildly/moderately disabled students (about two to four percent difference in GLA 
equal to or above enrolled grade), their counterparts in the severely disabled category showed 
slight (one to three percent) “decline.” Also in all, GLA in Mathematics was consistently 


somewhat higher than GLA in English Language Arts in both mentioned groups of students, for 
both reporting periods. 
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Table 32 — Difference in GLA Results: Matched Coded Students for 2006-07 - 2007-08 


2006-07 2007-08 


: English : English 
Language Arts manguaee Arts 


Students with Severe Disabilities 


[Silents with Severebiais 

a 

fcuaserecwoesgwie [Toa [oe fo fs fos fn [os Ps 
209 22 


All Grades 


ee 

[GtAtwoweniedgaie | 2 _[ » [1s [7 [oe | 2 
[otAdteveewotedgaie [70 |» [os] [use | oe 
fauana—SSCSC*id ts ‘Ps Po | fo fe fo [| 
a 
fauana——SSSCSi (| mew | os | os | aw | uso | a7 | asl 
Prot te | se [00 | ass | son | oats | 100 [oes | 
MM eee es 
FGLA equaloenollederde | ma | soa | mai | som | sco | rum | no | «707 | 
FGLAsboveewoledgnde | os |e [oi [9 [iw [us| io | «| 
Prot | tae [00 | name [sn [aa | 100 | aa | 


On the other hand, the overwhelming majority of students coded as “gifted” were at or above the 
enrolled grade level (see Table 32). To no surprise, these cumulative percentages were 
somewhat (three to five percent) higher than those for the non-coded students (seeTable 31). 
There were notable “spikes” (six to eight percent) in proportions of gifted students at above 
enrolled grade level in both Mathematics and English Language Arts in 2007-08 compared to 
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2006-07. This could be symptomatic of progress in this particular group of students (possibly 
due to more successful programming), but the current preliminary findings should be further 
substantiated by future collections of trend GLA data. 


Finally, general GLA results do not point to drastic differences between Canadian and foreign- 
born ESL students (see Table 32). Both groups were consistently less successful in English 
Language Arts in both reported school years than in Mathematics. At the same time, there is 
evidence of some progress in 2007-08 compared to 2006-07 in both subjects, for both groups of 
ESL students (cumulative percentage of students with GLA equal or above the enrolled grade). 
As already mentioned, these preliminary findings, while encouraging, need further confirmation 
using additional complete trend data. 


The data presented in Tables 31 and 32 illustrate the potential of using systematically collected 
GLA (grade level of achievement) data to identify trends and diagnose issues related to the 
programming for and performance of different groups of students. For example, the current 
preliminary results point to the importance of addressing various language deficiencies in both 
ESL and severely/moderately/mildly disabled students. Addressing language and literacy issues 
could boost achievement in other fields, including Mathematics. In addition, virtually “equal” 
general grade progress among some groups of students coded as severely disabled and 
mildly/moderately disabled raise questions regarding the adequacy and consistency of the coding 
standards and procedures across the system. Depending on the specific diagnostic or 
intervention objectives the continuous GLA data is a comprehensive corollary complementary to 
PAT data. 


Although trend grade-based analysis of the data on coded students is outside of the scope of this 
report, it is worth mentioning that future examination using fuller, systematically gathered GLA 
data may yield valid insights into issues affecting student achievement and point to potential 
solutions. 


GLA by PAT Analysis-Comparisons Using Achievement Levels 


In order to illuminate the relationship between the GLA data and provincial achievement test 
(PAT) data, PAT data was re-coded into the categories of “acceptable or excellence” and “below 
acceptable” and “excused or absent” in order to be comparable to the GLA categories of “at or 
above grade level” and “below grade level or GLA NA.” The groupings were chosen based on 
the current Alberta Education standard for cohort reporting. The groups were then cross- 
tabulated with the expectation that students who score at or above the acceptable level on PATs 
tend to be at or above grade level, and likewise those that score below acceptable tend to be 
below grade level. Tables 33 and 34 display these relationships with 76 to 81 percent of the 
students in Language Arts and 65 to 80 percent of the students in Mathematics who are at grade 
level are also at or above the acceptable level on the PATs. The 2007-08 data compares to the 
2006-07 data where 76 to 78 percent of the students in Language Arts and 66 to 80 percent of the 
students in Mathematics who were at grade level were also at or above the acceptable level on 
the PATs. 
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Table 33 - Comparison of English Language Arts PAT and GLA 


Grade Level of Achievement — English Language Arts 


At or Above Below Grade Total 
Grade Level Level or GLA 
NA 
Percent Total Percent Total Percent Total 
PAT -Grade3 Accept. or Excellence 78.7 29,711 512, 1,953 83.9 31,664 
English 
Language Arts rem Excused Ot gi. 325) 995 28347) abd | G08s 
Total 87.3 7 32,962° “127.- 4,787. 100%. 37.749 
PAT-Grade6 Accept. or Excellence _ 80.8 31,819 43 1,679 85.1 33,498 
English : 
Language Arts rie amie Excussdor 77060 «3027 «0 72s, 8AS 495,872 
Total 88.5 34,846 11.5 4,524 100.0 39,370 
PAT-—Grade9 Accept.orExcellence 76.4 «30,678 «= 2.1 859s 78S 31,537 
English 


Below Accept., Excused or 


Language Arts Absent 13.0 5.232 8.5 3,404 215 8,636 


Total 89.4 35,910 10.6 4,263 100.0 40,173 


Note: Bolded numbers represent consistent relationships between GLA and PAT data; underlined numbers indicate 
inconsistent relationships. 


It should be noted, in reviewing the above two tables and the data from 2006-07 that more 
students are categorized as “below grade level” in the PAT results than is true in GLA ratings. 
This suggests that in terms of evaluating acceptable progress, the PAT is a more difficult 
standard to attain than is the GLA. The two assessments can be expected to demonstrate some 
variance because they are different forms of assessment, but both are designed to assess whether 
a student has met grade level standards. The variance between these two sets of measures may 
be due to a number of possible explanations, such as: 1) it may be more difficult for teachers to 
assign a “below grade level” evaluation to their students than is the case for markers of the PAT 
assessments who do not know the students whose work they are judging; 2) student performance 
on PATs may be attenuated by test anxiety; 3) students may perform better on many assessments 
over time than on a single paper and pencil test; or 4) a combination of these factors and/or other 
factors. The key opportunity this analysis presents lie in identifying where discrepancies occur, 
both individually and for various student groups, and investigating the specific reasons for the 
discrepancy. Such reflection will help to ensure students and parents are well-served by the 
overall assessments used to judge student achievement. A priori pronouncements on the 
appropriate relationship between classroom assessment results and large scale test results ignore 
the inherent complexities associated with student assessment. 
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Table 34 - Comparison of Mathematics PAT and GLA 


Grade Level of Achievement — Mathematics 


At or Above Below Grade Level or Total 
Grade Level GLA NA 
Percent Tota: Percent Total Percent Total 
PAT~Grade3 Acceptor Excellence 0796 30067 .0e. De aos7 82.4 31,124 
Soe Siete 112 4.4219 6.4 2,406 17.6 6,625 
Total 90.8 34,286 9.2 3,463 100.0 37,749 


PAT-Grade6 Accept. or Excellence 76.4 30,075 5778.3 30,832 
Mathematics : 
Below Accept., 
Hannerdor Abusat 13.2 3,211 8.5 3,327 21:7 8,538 
Total 89.6 35,286 10.4 4,084 100.0 39,370 


PAT —Grade9 Accept or Excellence 648 26047. 170 699. 66.6 26,746 
Mathematics 
Below Accept., 
ec ol ae Wheat 0.7 8,321 12.7 5,106 33.4 13,427 
Total 85.5 34,368 14.5 5,805 100.0 40,173 


Note: Bolded numbers represent consistent relationships between GLA and PAT data; underlined numbers indicate 
inconsistent relationships. 


The key issue emerging from this analysis is to understand the implications for teachers and 
administrators who are experiencing significant variance between these two sets of measures. 
Should teachers be encouraged to teach more specifically to the test blueprints that underlie the 
PATs, or should they be encouraged to align their classroom assessment strategies more directly 
to the general and specific learning outcomes defined in the subject program of studies? 
Research on this question tends to suggest that the latter strategy has the most potential to ensure 
that when students are assessed with external tests that the results correspond to the classroom 
assessment results [see for example, /ntegrating Differentiated Instruction and Understanding by 
Design (Tomlison and McTighe, 2006)] for a comprehensive discussion of the dynamic 
relationship between pedagogy and assessment. 


According to classical test theory (Crocker and Algina, 1986) all assessments are subject to some 
degree of test error. Given this fact the more achievement data available the clearer the true 
picture of a student’s achievement becomes. Ore would expect some differences as well as 
complementary relationships in the designation of individuals in the two ratings provided by 
PAT and GLA data. Classroom assessment is based on an array of assessment methods over 
time for the GLA rating, ideally measuring the full and complete range of learning outcomes; 
whereas the PAT is a single, highly valid and reliable, selected and constructed response test that 
typically measures between 66 percent and 95 percent of the curricular outcomes in the tested 
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subjects*. The PATs are very likely the single best criterion-referenced assessment instrument 
available to the classroom teacher. Since the objective of both methods is to measure and provide 
evidence on how well a student is achieving as compared to the learning outcomes in the 
program of studies, one would expect a generally positive relationship between the students 
identified as “below” by both methods, and that is what we see in this data set. Where 
anomalous or inconsistent relationships are observed it presents the occasion to ask why and to 
delve deeper into the data to help understand why the measures differ. When such variance is 
present there is no substitute for careful review and reflection on the reasons for the discrepancy 
so that the student and parents are more fully informed of what the assessment results imply for 
future program decisions. 


Discussion and Conclusions 


Highlights of Findings 

The key question addressed by this report considered how GLA data contributes to our 
knowledge base in terms of provincial level student achievement overall and examined the data 
in relationship to specific student cohorts who are served by special programming. Another 
important issue is demonstrating the potential of applied research and the practical utilities of 
systematically collected GLA data, which, in conjunction with other data and information 
sources, can be applied in the decision-making process at a provincial, jurisdiction and school 
level. Lastly, this report may serve to give parents and the public a clearer picture of student 
achievement in Alberta. 


The error rate for 2007-08 GLA data submission was low with only 0.014 percent of the files 
submitted having initial errors. This low error percentage attests to a significant progress in the 
refinement of the data collection and reporting processes. The quality of data collected from 
schools is expected to further improve after the error detection software is introduced in the data 
collection process in the near future as a component of the “Provincial Approach to Student 
Information” initiative. Systematic collection of full sets of GLA data will make it possible to 
conduct trend analysis of achievement for different groups of students at the provincial, 
jurisdiction and school levels. 


As expected and similar to the 2006-07 analysis, the 2007-08 data demonstrates less variation for 

the total cohort than for specific sub-groupings of student achievement. The following key 

findings emerged from the analysis of 2007-08 GLA data as well as from the examination of the 
available starting two-year trend data: 

e Substantial variations continue to be observed in GLA between major student sub-groups. 
For example, over half (55 to 63 percent) of students coded with severe or mild/moderate 
disabilities and between 77 and 86 percent of Canadian and foreign-born ESL students 
achieved at or above grade level in English Language Arts and Mathematics compared to 92 
to 94 percent of non-coded students and 98 to 99 percent of gifted students (refer to Tables 5, 
6, 9, 12, 13 and 14). 


e In addition, the following differences were observed within some of these sub-groups: 


* These figures were obtained from a discussion paper presented to the Program and Accountability Advisory 
Committee, November 8, 2007. 
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- The percentages of students attaining at or above grade level varied substantially by 
specific types of severe or mild/moderate disability. 

- Congruent with previously collected (2006-07) GLA data, nearly three times as many 
males as females were coded as severely disabled and almost twice as many males were 
coded as mildly/moderately disabled compared to females (see Tables 8 and 11). At the 
same time, males coded with different types of disabilities generally (with a few 
exceptions) tended to outperform coded females on GLA. 

- Coded groups of students (especially those with mild/moderate disabilities and Canadian 
and foreign born ESL) tended to perform better in Mathematics than in English Language 
Arts, 

- Students coded as gifted were also more likely to achieve better in Mathematics. A much 
higher percentage of students coded as gifted continue to be assessed above grade level in 
Mathematics than in English Language Arts or French Language Arts. Available trend 
data over two years (see Table 32) points to higher percentages of gifted students 
performing above their enrolled grade levels in 2007-08 compared to 2006-07, both in 
English Language Arts and Mathematics. This could be symptomatic either of improved 
programming and more appropriate grade placement of these students. 

- When comparing foreign-born and Canadian-born ESL students, both groups of students 
performed at a similar level in Mathematics. While during previous, 2006-07 GLA 
reporting period Canadian-born ESL students appeared to be at an advantage in English 
Language Arts, the currently available two-year trend data did not confirm this finding. 
Recent 2007-08 data indicate similar percentages of Canadian and foreign-born ESL 
students achieving at grade level in English Language Arts, and may indicate foreign- 
born ESL students are closing the gap with their Canadian-born counterparts. 

- While slightly higher percentages of mildly or moderately disabled students tended to 
achieve at or above grade level in 2007-08 compared to 2006-07, their severely disabled 
counterparts showed a reversed trend (see Table 32). However, the comparative trend 
analysis was based on a partial data set (60 percent of schools reported GLA in 2006-07, 
which also embraced only two consecutive years of data collection. 


Congruent to findings based on previously collected (2006-07) data, females slightly 
outperformed males on GLA (three to five percent more were at or above grade level 
compared to males). 


The percent of students below grade level tended to increase through the elementary grades 
and peaked at Grades 8 and 9. The two-year trend data (Figures 7 and 8) revealed subject- 
based variations in this pattern. While the percentage of students below grade level tended to 
increase with grade for Mathematics, the proportion of these students tended to decrease with 
grade for English Language Arts. This finding is compatible with other quantitative and 
qualitative evidence pointing to the issues related to achievement in Mathematics in junior- 
high school. 


By analogy with the results obtained using past GLA data, the negative effect of high 
mobility on student achievement was evident in 2007-08 GLA data. The differences between 
highly and low mobile students with GLA below grade level ranged from six to over 10 
percent, depending on a grade and subject. 
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e The age effect? was apparent in English Language Arts 2007-08 GLA data, especially in 
Grades 1 and 2. After Grade 3 the age effect tapered off. This relationship also unfolded in a 
similar way in the 2006-07 GLA data, with the only difference that it was more notable for 
Grade 3 and was also evident up to Grade 5. These preliminary trend results confirm that the 
age effect is most likely to occur in early elementary grades. 


e The results of comparisons between GLA and PAT outcomes for Grades 3, 6 and 9 bear 
much similarity to the findings in the previous GLA reporting period. Overall, there was 
almost 80 percent alignment between students assessed at or above grade level of 
achievement and earning acceptable or excellence grade on provincial achievement tests. 
However, the previously observed large difference between Math 9 Provincial Achievement 
Test data and GLA (only 65 percent congruence) was also observed in the recent, 2007-08 
GLA data. This phenomenon warrants further trend observations and explanations of why 
such large gaps between GLA and PAT assessment results tend to occur in Grade 9 Math. 


Opportunities for Enhanced Dialogue on Student Achievement 


GLA data make it possible to systematically obtain consistent information on student 
achievement, which otherwise would not be available. PAT results represent high quality but 
more periodic data on student achievement and academic progress. GLA provides important 
consistent information that would not otherwise be available for students in grades not tested by 
PATs. GLA data can be applied as a useful supplement to consideration of PAT outcomes, and 
can be used on its own in grades not tested with large-scale assessments to interpret patterns and 
trends in student achievement, depending on context and/or decision-making needs. 


This report points to a number of potential questions that may benefit from further reflection 
within schools, jurisdictions, stakeholder organizations and Alberta Education. For example: 


e Why is the proportion of male students coded with a severe disability so much higher than 
female students? Also, why do females coded with severe as well as mild/moderate 
disabilities tend to perform at a somewhat lower level on GLA compared to males? 

e Why are gifted students more likely to have a GLA above their enrolled grade in 
Mathematics than in Language Arts 

e What strategies might be most effective in offsetting the negative effect of mobility on 
student achievement? 

e What pedagogical adaptations might be helpful in mitigating the age effect in the early 
grades? 

e Does the Grade 9 PAT in Mathematics or the GLA for Mathematics 9 represent the best 
predictive validity for subsequent high school math achievement? 


The ensuing practical implications based on GLA data analysis include, but are not limited to the 
following examples: 


> Age effect is defined as older students in a grade tending to have higher average PAT test scores than the younger students in 
that same grade (Alberta Learning, 2001). 
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Similar general grade progress among some groups of students coded as severely disabled 
and mildly/moderately disabled may be indicative of a lack of consistency in the coding 
standards and procedures across the system and reflects the complexity of the learning 
profiles of individual students. These patterns support the current provincial initiative 
targeting a review of current research and best practices and input from Albertans in order to 
build a more effective special education policy framework. 


Relatively better progress by males, who represent the vast majority of the student population 
coded with various types of disabilities in comparison to females, may be due to the lack of 
special programming and attention to the needs of relatively fewer coded females or perhaps a 
phenomenon of over-coding of male students. 


The observed higher degree of success in Mathematics GLA in students coded as ESL or 
gifted in comparison to English Language Arts may be due to a number of factors. For 
example, it could be symptomatic of various language (learning) deficiencies among ESL 
students which support the need for an enhanced focus on language and literacy. The 
observed discrepancies for gifted students may be due more to curricular design issues as 
opposed to ability levels of these students. 


The comparisons of GLA data with PAT results (refer to Tables 33 and 34) may represent a 
useful exercise for identification and examination of issues that would otherwise be 
concealed if only PAT data were taken into consideration. For example: 


- The “discrepancies” in GLA and PAT results could point to issues such as inflated 
classroom grading or sources of internal and/or external bias in the provincial 
achievement tests. However, it would be useful to try to gain a more detailed 
understanding of why some students who “passed” GLA fall below the acceptable 
standard on PATs or vice-versa. There could be various underlying issues that explain 
non-congruence between GLA and PAT results. 

- Given that lower than expected student achievement in Mathematics have been a 
persistent issue in junior and senior high grades, it is imperative to uncover the reason for 
a large inconsistency between GLA and PAT results for Grade 9 Mathematics (see Table 
34). This would include answering the earlier mentioned question regarding the 
predictive validity of both GLA and PAT assessment in Grade 9 for subsequent high 
school math achievement. 

- GLA data is valuable for both trend and comprehensive examination of student 
achievement complementary to PAT results as useful check points. 


Finally, collection and analysis of GLA data raises general questions about assessment theory 
and practices, including standards and comparability of GLA data. Since teaching practices 
vary a lot in terms of methods and practices (that can depend on individual groups of 
students), it would be difficult and undesirable to “uniformly standardize” GLA assessment 
techniques. At the same time, it would be useful to support excellent assessment approaches 
and methodologies that would improve comperability and consistency of GLA data coming 
from different sources. This would require consistent professional development training and 
preparation in assessment for already practicing and future teachers. The concurrent Alberta 
Student Assessment Study will provide insight to this issue. 
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Future Data Collection and Analysis 


Additional analysis of GLA data in relationship to 2006 census data will be available later in the 
current school year to provide an indication of the extent to which variables external to schools 
influences student achievement. To create the most accurate comparison of the census data and 
the GLA data, the census data will be broken down by enrolled students’ postal code. 


Examples of Socio-economic status (SES) variables include Mother’s Level of Education, 
Average Family Income/LICO (Low-Income Cut Off) and Home Rental/Ownership. 
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